Journal of Pathology Informatics

BackgroundThe global rise in endometrial cancer, including in Japan, and the shortage of pathologists and cytotechnologists has increased the diagnostic burden, emphasizing the need for AI-based diagnostic support model using deep learning. This study aims to advance an existing AI-supported endometrial cytology model for clinical application. MethodsWe compared two datasets--one annotated for both benign and malignant clusters, and one for malignant only--using YOLOv5x and YOLOv7 models evaluated by mAP. We also assessed the correlation between AI diagnostic accuracy and the level of difficulty perceived by human diagnosticians using the Two One-Sided Tests (TOST) procedure. Additionally, we applied Grad-CAM to visualize and enhance the interpretability of the AI models decision-making process. ResultsThe YOLOv5x model with both benign and malignant annotations achieved the highest malignant mAP at 0.798 compared to Yolov7. The TOST analysis showed no significant difference in perceived diagnostic difficulty between cases that were correctly and incorrectly diagnosed by the AI model, indicating consistent AI accuracy regardless of case difficulty. Grad-CAM visualizations clarified the AI models decision-making basis; in some cases, the model appeared to focus on regions different from those typically attended to by human diagnosticians. ConclusionThe AI support model showed high and consistent accuracy in endometrial cytology, regardless of diagnostic difficulty as perceived by human diagnosticians. Grad-CAM visualizations revealed diagnostic patterns, with AI occasionally focusing on regions different from those emphasized by human diagnosticians. This study advances the real-time, microscope-integrated AI system toward clinical application.

Well-annotated exemplars are an important prerequisite for supervised deep learning schemes. Unfortunately, generating these annotations is a cumbersome and laborious process, due to the large amount of time and effort needed. Here we present a deep-learning-based iterative digital pathology annotation tool that is both easy to use by pathologists and easy to integrate into machine vision systems. Our pathology image annotation tool greatly reduces annotation time from hours to a few minutes, while maintaining high fidelity with human-expert manual annotations. Here we demonstrate that our active learning tool can be used for a variety of pathology annotation tasks including masking tumor, stroma, and lymphocyte-rich regions, among others. This annotation automation system was validated on 90 unseen digital pathology images with tumor content from the CAMELYON16 database and it was found that pathologists gold standard masks were re-produced successfully using our tool. That is, an average of 2.7 positive selections (mouse clicks) and 8.0 negative selections (mouse clicks) were sufficient to generate tumor masks similar to pathologists gold standard in CAMELYON16 test WSIs. Furthermore, the developed image annotation tool has been used to build gold standard masks for hundreds of TCGA digital pathology images. This set was used to train a convolutional neural network for identification of tumor epithelium. The developed pan-cancer deep neural network was then tested on TCGA and internal data with comparable performance. The validated pathology image annotation tool described herein has the potential to be of great value in facilitating accurate, rapid pathological analysis of tumor biopsies.

Pancreatic ductal adenocarcinoma has some of the worst prognostic outcomes among various cancer types. Detection of histologic patterns of pancreatic tumors is essential to predict prognosis and decide about the treatment for patients. This histologic classification can have a large degree of variability even among expert pathologists. This study proposes a graph convolutional network-based deep learning model to detect aggressive adenocarcinoma and less aggressive pancreatic tumors from benign cases. Our model uses a convolutional neural network to extract detailed information from every small region in a whole-slide image. Then, we use a graph architecture to aggregate the extracted features from these regions and their positional information to capture the whole-slide level structure and make the final prediction. We evaluated our model on an independent test set and achieved an F1 score of 0.85 for detecting neoplastic cells and ductal adenocarcinoma, significantly outperforming other baseline methods. If validated in prospective studies, this approach has a great potential to assist pathologists in identifying adenocarcinoma and other types of pancreatic tumors in clinical settings.

ContextBreast cancer is one of the most common cancers in women. With early diagnosis, some breast cancers are highly curable. However, the concordance rate of breast cancer diagnosis from histology slides by pathologists is unacceptably low. Classifying normal versus tumor breast tissues from microscopy images of breast histology is an ideal case to use for deep learning and could help to more reproducibly diagnose breast cancer. Since data preprocessing and hyperparameter configurations have impacts on breast cancer classification accuracies of deep learning models, training a deep learning classifier with appropriate data preprocessing approaches and optimized hyperparameter configurations could improve breast cancer classification accuracy. Methods and MaterialUsing 12 combinations of deep learning model architectures (i.e., including 5 non-specialized and 7 digital pathology-specialized model architectures), image data preprocessing, and hyperparameter configurations, the validation accuracy of tumor versus normal classification were calculated using the BreAst Cancer Histology (BACH) dataset. ResultsThe DenseNet201, a non-specialized model architecture, with transfer learning approach achieved 98.61% validation accuracy compared to only 64.00% for the digital pathology-specialized model architecture. ConclusionsThe combination of image data preprocessing approaches and hyperparameter configurations have a profound impact on the performance of deep neural networks for image classification. To identify a well-performing deep neural network to classify tumor versus normal breast histology, researchers should not only focus on developing new models specifically for digital pathology, since hyperparameter tuning for existing deep neural networks in the computer vision field could also achieve a high (often better) prediction accuracy.

The evaluation of erb-b2 receptor tyrosine kinase 2 (ERBB2 or HER2) gene amplification status through Dual in Situ Hybridization (DISH) currently relies on manual assessment by pathologists. There are several deep learning-based algorithms for H&E or ISH analysis. However, DISH analysis tools are still lacking. We developed a fully automated deep learning-based quantification system to assist pathologists in identifying the most relevant cells throughout the entire DISH image. In the comparison between pathologists and the auto-quantification system, the overall percentage agreement (OPA) by case was 88. 9% (80/90). These results demonstrate that each image, with a processing time of approximately 1 minute, achieves similar results compared to pathologists assessments, while the manual procedure will take 10-20 times longer to examine the same specimen. This approach offers a versatile system for bright-field HER2 DISH image analysis. The system provides faster, cheaper, standardized, and versatile diagnostic tools to aid pathologists in the HER2 DISH diagnostic process.

Non-alcoholic steatohepatitis (NASH) is a liver disease characterized by excessive lipid accumulation and disease progression is typically assessed through inspection of a Trichrome stain for Fibrosis staging. As the public health burden of NASH worsens due to evolving lifestyle habits, pathology laboratory resources will become increasingly strained due to rising demand for specialized stains. Virtual staining processes, computational methods which can synthesize the application of chemical staining reagents, can potentially provide resource savings by obviating the need to acquire specialized stains. Virtual staining technologies are assessed by comparing virtual and real tissue stains for their realism and ability to stage. However, these assessment methods are rife with statistical mistreatment of observed phenomena that are difficult to account for. Bridge category ratings represent a phenomenon where a pathologist may assign two adjacent stages simultaneously, which may bias and/or reduce the power of research findings. Such stage assignments were frequently reported in a large-scale assessment of Virtual Trichrome technologies yet were unaccounted for since no statistical adjustment procedures existed. In this work, we provide an updated assessment of Virtual Trichrome technologies using Bridge Category Models, which account for these bridge ratings. We report that two of four pathologists tended to assign lower Fibrosis stages to virtually stained tissue while the other two pathologists assigned similar stages. These research findings differ when bridge ratings are not accounted for. While promising, these results indicate further room for algorithmic finetuning of Virtual Trichrome technologies.

BackgroundDigital pathology significantly improves diagnostic efficiency and accuracy; however, pathological tissue sections are scanned at high resolutions (HR), magnified by 40 times (40X) incurring high data volume, leading to storage bottlenecks for processing large numbers of whole slide images (WSIs) for later diagnosis in clinic and hospitals. MethodWe propose to scan at a magnification of 5 times (5X). We developed a novel multi-scale deep learning super-resolution (SR) model that can be used to accurately computes 40X SR WSIs from the 5X WSIs. ResultsThe required storage size for the resultant data volume of 5X WSIs is only one sixty-fourth (less than 2%) of that of 40X WSIs. For comparison, three pathologists used 40X scanned HR and 40X computed SR WSIs from the same 480 histology glass slides spanning 47 diseases (such tumors, inflammation, hyperplasia, abscess, tumor-like lesions) across 12 organ systems. The results are nearly perfectly consistent with each other, with Kappa values (HR and SR WSIs) of 0.988{+/-}0.018, 0.924{+/-}0.059, and 0.966{+/-}0.037, respectively, for the three pathologists. There were no significant differences in diagnoses of three pathologists between the HR and corresponding SR WSIs, with Area under the Curve (AUC): 0.920{+/-}0.164 vs. 0.921{+/-}0.158 (p-value=0.653), 0.931{+/-}0.128 vs. 0.943{+/-}0.121 (p-value=0.736), and 0.946{+/-}0.088 vs. 0.941{+/-}0.098 (p-value=0.198). A previously developed highly accurate colorectal cancer artificial intelligence system (AI) diagnosed 1,821 HR and 1,821 SR WSIs, with AUC values of 0.984{+/-}0.016 vs. 0.984{+/-}0.013 (p-value=0.810), again with nearly perfect matching results. ConclusionsThe pixel numbers of 5X WSIs is only less than 2% of that of 40X WSIs. The 40X computed SR WSIs can achieve accurate diagnosis comparable to 40X scanned HR WSIs, both by pathologists and AI. This study provides a promising solution to overcome a common storage bottleneck in digital pathology.

Renal oncocytoma and chromophobe renal cell carcinoma are two kidney cancer types that present a diagnostic challenge to pathologists and other clinicians due to their microscopic similarities. While RO is a benign renal neoplasm, ChRCC is considered malignant. Therefore, the differentiation between the two is crucial. In this study, we introduce an explainable framework to accurately differentiate ChRCC from RO, histologically. Our approach examined H&E-stained images of 656 ChRCC and 720 RO, and achieved a diagnostic accuracy of 88.2%, the sensitivity of 87%, and 100% specificity for explainable AI, which either outperforms or operate on par with convolutional neural network (CNN) models. Besides, we enrolled 44 pathology experts (including pathologists and pathology trainees) to differentiate the two tumors. The average accuracy of pathologists was 73%, which is 15.2% lower than our framework. These results indicate that the combination of human expert along with explainable AI achieve higher accuracy in differentiating the two tumors, while it reduces the workload of experts and offers the desired explainability for the medical experts.

The rapid advancement of precision medicine and personalized healthcare has heightened the demand for accurate diagnostic tests. These tests are crucial for administering novel treatments like targeted therapy. To ensure the widespread availability of accurate diagnostics with consistent standards, the integration of computer-aided systems has become essential. Specifically, computer-aided systems that assess biomarker expression have thrusted through the widespread application of deep learning for medical imaging. However, the generalizability of deep learning models has usually diminished significantly when being confronted with data collected from different sources, especially for histological imaging in digital pathology. It has therefore been challenging to effectively develop and employ a computer-aided system across multiple medical institutions. In this study, a biomarker computer-aided framework was proposed to overcome such challenges. This framework incorporated a new approach to augment the composition of histological staining, which enhanced the performance of federated learning models. A HER2 assessment system was developed following the proposed framework, and it was evaluated on a clinical dataset from National Taiwan University Hospital and a public dataset coordinated by the University of Warwick. This assessment system showed an accuracy exceeding 90% for both institutions, whose generalizability outperformed a baseline system developed solely through the clinical dataset by 30%. Compared to previous works where data across different institutions were mixed during model training, the HER2 assessment system achieved a similar performance while it was developed with guaranteed patient privacy via federated learning.

ContextIn digital pathology, blurriness in whole slide images (WSI) is a common issue, with severe blurriness widely acknowledge as a critical factor that can degrade the performance of artificial intelligence (AI) models. However, the effects of the typical levels of blurriness observed in real-world pathological images on the robustness of AI predictions remains unclear and unexplored. ObjectiveTo evaluate the impact of WSI blurring on the robustness of AI prediction in real-world setting. DesignA retrospective study was conducted using 8,000 WSIs and corresponding AI predictions from four AI models trained on data from two scanners and two organs. WSIs were categorized into concordant and discordant groups based on AI-prediction accuracy. Analyses included: 1) comparing blur metrics between groups, 2) determining the odds ratio between the proportions of blurry patch in WSIs and prediction concordance, and 3) assessing model performance across varying blur intensities. ResultsFor each organ-scanner pair, the average wavelet score and Laplacian variance for WSIs between the two groups did not show a statistically significant difference model (p > 0.05 for both metrics), except for one, and their effect sizes were small (Cohens D < 0.2 for both metrics). Additionally, no statistically significant association was observed between AI prediction concordance and the proportion of blurry images in WSIs (confidence intervals included 1, respectively). Model performance remained robust even at high blur level (radius=1) at which patch image had Laplacian variance of 162.88 and a wavelet score of 1880.07, corresponding to the top 1.22% and 2.16% of blurriness respective, in our dataset. ConclusionsThe findings empirically suggest that the typical levels of WSI blurriness encountered in real-world settings may not significantly compromise the robustness of AI predictions.

Image registration involves finding the best alignment between different images of the same object. In these tasks, the object in question is viewed differently in each of the images (e.g. different rotation or light conditions, etc.). In digital pathology, image registration aligns correspondent regions of tissue from different stereotactic viewpoints (e.g. subsequent deeper sections of the same tissue). These comparisons are important for histological analysis and can facilitate previously unavailable manipulations, such as 3D tissue reconstruction and cell-level alignment of immunohistochemical (IHC) and special stains. Several benchmarks have been established for evaluating image registration techniques for histological tissue; however, little work has evaluated the impact of scaling registration techniques to Giga-Pixel Whole Slide Images (WSI), which are large enough for significant memory limitations, and contain recurrent patterns and deformations that hinder traditional alignment algorithms. Furthermore, as tissue sections often contain multiple, discrete, smaller tissue fragments, it is unnecessary to align an entire image when the bulk of the image is background whitespace and tissue fragments orientations are often agnostic of each other. We present a methodology for circumventing large-scale image registration issues in histopathology and accompanying software. By removing background pixels, parsing the slide into discrete tissue segments, and matching, orienting and registering smaller segment pairs, we recovered registrations with lower Target Registration Error (TRE) when compared to utilizing the unmanipulated WSI. We tested our technique by having a pathologist annotate landmarks from 13 pairs of differently stained liver biopsy slides, performing WSI and segment-based registration techniques, and comparing overall TRE. Preliminary results demonstrate superior performance of registering segment pairs versus registering WSI (difference of median TRE of 44 pixels, p<0.001). Segment matching within WSI is an effective solution for histology image registration but requires further testing and validation to ensure its viability for stain translation and 3D histology analysis.

Integration of digital pathology (DP) into clinical diagnostic workflows is increasingly receiving attention as new hardware and software become available. To facilitate the adoption of DP, the Swiss Digital Pathology Consortium (SDiPath) organized a Delphi process to produce a series of recommendations for DP integration within Swiss clinical environments. This process saw the creation of 4 working groups, focusing on the various components of a DP system (1) Scanners, Quality Assurance and Validation of Scans, (2) Integration of WSI-scanners and DP systems into the Pathology Laboratory Information System, (3) Digital Workflow - compliance with general quality guidelines, and (4) Image analysis (IA)/artificial intelligence (AI), with topic experts for each recruited for discussion and statement generation. The work product of the Delphi process is 83 consensus statements presented here, forming the basis for "SDiPath Recommendations for Digital Pathology". They represent an up-to-date resource for national and international hospitals, researchers, device manufacturers, algorithm developers, and all supporting fields, with the intent of providing expectations and best practices to help ensure safe and efficient DP usage.

AO_SCPLOWBSTRACTC_SCPLOWWith the rise of digital pathology, integrating digital slides with deep learning-based decision support systems is becoming increasingly common in clinical practice. Tissue region segmentation which is distinguishing tissue from background/artefacts, is an important pre-requisite in many digital pathology pipelines both for the laboratories as their first step in digitalizing the glass slides of tissue samples and turning them to whole slide images (WSIs) using scanners, and also for DL researches such as region-of-interest cropping, tumor detection, cell segmentation. However, it is well known that WSI scanners can fail in detecting all tissue regions, due to the tissue type, or due to weak staining and this is because of their not robust enough tissue detection algorithms which makes segmentation of WSIs a challenging task. Hence, this study develops a fast, lightweight, accurate, CPU-ready DL approach, enabling fast and reliable tissue region segmentation model by training and testing it across seven different institutional H&E and IHC stained WSIs to result a strong in generalization with the 22 to 56 s/WSI inference time using CPU that markedly outperforms classical OTSU thresholding, particularly in preserving challenging or faint tissue regions by achieving notably higher and more consistent performance than OTSU, with median Jaccard and Dice scores of approximately 0.86 and 0.92, respectively, compared to OTSU whcih was between 0.56 and 0.72. Our approach provides a practical, open-source solution for resource-limited pathology settings. We publicly released dataset obtained from Bahcesehir Medical School, and code to foster benchmarking and further advances in efficient, deployable computational pathology. The model could be used in digital slide scanners to improve the scanning process and in the pre-processing stages of DL pipelines to prepare high-quality datasets.

Recent advances in the field of pathology coupled with the rapid evolution of machine learning based techniques have revolutionized healthcare practices. Colorectal cancer accounts for one of the top 5 cancers with high incidence (126,240 in 2020) with a high mortality worldwide [1] [2]. Tissue biopsy remains to be the gold standard procedure for accurate diagnosis, treatment planning and prognosis prediction [3]. As an image based modality, pathology has attracted a lot of attention for development of AI algorithms and there has been a steady increase in the number of filings for FDA authorized use of AI algorithms in clinical practice [4]. The SemiCOL Challenge aims to develop computational pathology methods for automatic segmentation and classification of tumor and other tissue classes using H&E stained images. In this paper, we present a novel machine learning framework addressing the SemiCOL Challenge, focusing on semantic segmentation, segmentation-based whole-slide image classification, and effective use of limited annotated data. Our approach leverages deep learning techniques and incorporates data augmentation to improve the accuracy and efficiency of tumor tissue detection and classification in CRC. The proposed method achieves an average Dice score of 0.2785 for segmentation and an AUC score of 0.71 for classification across 20 whole-slide images. This framework has the potential to revolutionize the field of computational pathology, contributing to more efficient and accurate diagnostic tools for colorectal cancer.

Advanced image processing methods have shown promise in computational pathology, including the extraction of crucial microscopic features from histology images. Accurate detection and classification of cell nuclei from whole-slide images (WSI) play a crucial role in capturing the molecular and morphological landscape of the tissue sample. They enable widespread downstream applications, including cancer diagnosis, prognosis, and discovery of novel markers. Robust nuclei detection and classification are challenging due to the high intra-class variability and inter-class similarity of the microscopic morphological features. This is further compounded by the domain shift arising due to the variability in tissue types, staining protocols, and image acquisition. Motivated by the ability of the recent deep learning techniques to learn complex patterns in a biasfree manner, we develop a novel and robust deep learning model TransNuc, based on vision transformers, for simultaneous detection and classification of cell nuclei from H&E stained WSI. We benchmarked TransNuc on the comprehensive Open Pan-cancer Histology Dataset (PanNuke), sampled from over 20,000 WSI, comprising 19 different tissue types and five clinically important cell classes, namely, Neoplastic, Epithelial, Inflammatory, Connective, and Dead cells. TransNuc exhibited superior performance compared to the state-of-theart, including Hover-Net and Micro-Net. TransNuc was able to learn robust feature representations and thereby perform consistently better for the abundant classes such as neoplastic, and the under-represented classes such as dead cells. Similar performance gains were also obtained for epithelial and connective classes that have a significant inter-class morphological similarity.

The spermatogenic stage serves as a vital criterion for assessing normal spermatogenesis and is central to evaluating reproductive toxicity. Current manual methods for spermatogenic stage evaluation are time-intensive, require expert knowledge, and are less effective in detecting subtle changes or comparing stage frequencies across samples. To overcome these limitations, this study introduces a method leveraging the object detection models, Region-based Convolutional Neural Networks (R-CNN), for efficient and accurate spermatogenic stage evaluation. 14 stages were identified using Periodic Acid-Schiff (PAS)-stained Sprague-Dawley (SD) rat testicular tissue, and the approach was further applied to atrophied testicular samples as a real-world example. The model achieved a mean average precision of 0.869 and a mean average recall of 0.977 in detecting spermatogenic stages and atrophy. Agreement with pathologist assessments exceeded 91%, providing objective benchmarks for stage evaluation and facilitating the comparison of stage frequencies across multiple samples. In atrophied tissues, the model enabled quantitative grading by analyzing proportional changes in atrophied seminiferous tubules relative to normal tubules. This automated approach reduces the workload of pathologists while delivering rapid and precise assessments of toxicological changes in spermatogenesis. By integrating deep learning models, this study enhances both the accuracy and efficiency of pathological evaluations, offering a transformative tool for reproductive toxicity studies.

Evaluation of a tissue biopsy is often required for the diagnosis and prognostic staging of a disease. Recent efforts have sought to accurately quantitate the distribution of tissue features and morphology in digitized images of histological tissue sections, Whole Slide Images (WSI). Generative modeling techniques present a unique opportunity to produce training data that can both augment these models and translate histologic data across different intra-and-inter-institutional processing procedures, provide cost-effective ways to perform computational chemical stains (synthetic stains) on tissue, and facilitate the creation of diagnostic aid algorithms. A critical evaluation and understanding of these technologies is vital for their incorporation into a clinical workflow. We illustrate several potential use cases of these techniques for the calculation of nuclear to cytoplasm ratio, synthetic SOX10 immunohistochemistry (IHC, sIHC) staining to delineate cell lineage, and the conversion of hematoxylin and eosin (H&E) stain to trichome stain for the staging of liver fibrosis.

BackgroundPanoptic segmentation networks are a newer class of image segmentation algorithms that are constrained to understand the difference between instance-type objects (objects that are discrete countable entities, such as renal tubules) and group-type objects (uncountable, amorphous regions of texture such as renal interstitium). This class of deep networks has unique advantages for biological datasets, particularly in computational pathology. MethodsWe collected 126 periodic acid Schiff whole slide images of native diabetic nephropathy, lupus nephritis, and transplant surveillance kidney biopsies, and fully annotated them for the following micro-compartments: interstitium, glomeruli, globally sclerotic glomeruli, tubules, and arterial tree (arteries/arterioles). Using this data, we trained a panoptic feature pyramid network. We compared performance of the network against a renal pathologists annotations, and the methods transferability to other computational pathology domain tasks was investigated. ResultsThe panoptic feature pyramid networks showed high performance as compared to renal pathologist for all of the annotated classes in a testing set of transplant kidney biopsies. The network was not only able to generalize its object understanding across different stains and species of kidney data, but also across several organ types. ConclusionsPanoptic networks have unique advantages for computational pathology; namely, these networks internally model structural morphology, which aids bootstrapping of annotations for new computational pathology tasks.

Digital pathology using whole slide imaging (WSI) and artificial intelligence (AI) has the potential to transform diagnostic workflows, but adoption remains limited by technical complexity and scalability. We developed the Computational Renal Pathology Suite (ComPRePS), a scalable cloud-based platform that automates WSI ingestion, compartmental segmentation, feature extraction, and AI-assisted interpretation through an integrated high-performance architecture. ComPRePS was evaluated in two use cases. First, using 213 procurement biopsies, we compared conventional assessments with automated AI analyses and a hybrid AI-assisted expert workflow. ComPRePS AI-assisted methods achieved higher precision and significantly improved interobserver agreement for key lesions, including global glomerulosclerosis, interstitial fibrosis and tubular atrophy, and arterial intimal thickening. Second, ComPRePS enabled high-throughput quantitative profiling of glomerular and tubular features across minimal change disease, diabetic nephropathy, and amyloid nephropathy revealing disease-specific phenotypic patterns inaccessible to manual evaluation. Overall, ComPRePS improves reproducibility, interpretability, and objectivity in renal pathology, bridging computation with clinical practice.

Segmentation of histology tissue whole side images is an important step for tissue analysis. Given enough annotated training data modern neural networks are capable accurate reproducible segmentation, however, the annotation of training datasets is time consuming. Techniques such as human in the loop annotation attempt to reduce this annotation burden, but still require a large amount of initial annotation. Semi-supervised learning, a technique which leverages both labeled and unlabeled data to learn features has shown promise for easing the burden of annotation. Towards this goal, we employ a recently published semi-supervised method: datasetGAN for the segmentation of glomeruli from renal biopsy images. We compare the performance of models trained using datasetGAN and traditional annotation and show that datasetGAN significantly reduces the amount of annotation required to develop a highly performing segmentation model. We also explore the usefulness of using datasetGAN for transfer learning and find that this greatly enhances the performance when a limited number of whole slide images are used for training.